Cross-model consensus of explanations and beyond for image classification models: an empirical study

Existing interpretation algorithms have found that, even deep models make the same and right predictions on image, they might rely different sets of input features for classification. However, among these features, some common be used by majority models. In this paper, we are wondering what various classification whether with better performance may favor those features. For purpose, our works uses an algorithm to attribute importance (e.g., pixels or superpixels) as explanations, proposes cross-model consensus explanations capture Specifically, first prepare a set committee, then deduce explanation every model, obtain across entire committee through voting. With conduct extensive experiments using 80+ 5 datasets/tasks. We find three interesting phenomena follows: (1) obtained from image is aligned ground truth semantic segmentation; (2) measure similarity result each model in (namely score), positive correlations between score performance; (3) coincidentally correlates interpretability.